Script Identification in Natural Scene Image and Video Frame using Attention based Convolutional-LSTM Network

摘要

Script identification facilitates many important applications indocument/video analysis. This paper focuses on the problem of scriptidentification in scene text images and video scripts. Because of low imagequality, complex background and similar layout of characters shared by somescripts like Greek, Latin, etc., text recognition in such scenario isdifficult. Most of the recent approaches usually apply a patch-based CNNnetwork with summation of obtained features, or only a CNN-LSTM network to getthe identification result. Some use a discriminative CNN to jointly optimizemid-level representations and deep features. In this paper, we propose a novelmethod that involves extraction of local and global features using CNN-LSTMframework and weighting them dynamically for script identification. First weconvert the images into patches and feed them into a CNN-LSTM framework.Attention-based patch weights are calculated applying softmax layer after LSTM.Then we do patch-wise multiplication of these weights with corresponding CNN toyield local features. Global features are also extracted from last cell stateof LSTM. We employ a fusion technique which dynamically weights the local andglobal features for an individual patch. Experiments have been done in twopublic script identification datasets, SIW-13 and CVSI2015. Our learningprocedure achieves superior performance compared with previous approaches.